As the principle method of obtaining a single high definition image by combining a plurality of images obtained using a plurality of imaging optical systems (multiple-lens imaging system), for example, the method disclosed in “Acquisition of Super High Definition Pictures” (Kiyoharu Aizawa et al., the Journal of the Institute of Image Electronics Engineers of Japan, 90-03-04, p 23 to 28) is known. According to this method, a single high definition image is obtained by superimposition of two images obtained using two imaging optical systems such that the pixels of one image are interposed between the pixels of the other image.
The same applies when this principle is applied to color images. Specifically, as the method of obtaining high definition color images, there are the following methods: one is to employ a plurality of imaging optical systems equipped with color filters arranged in the Bayer arrangement, a stripe arrangement or the like; and another is to employ a plurality of imaging optical systems having different chromatic sensitivity characteristics.
A typical multiple-lens image pickup device will be described with reference to FIG. 15. In FIG. 15, a plurality of imaging optical systems 101, 102 and 103 form object images on a plurality of image sensors 104, 105 and 106, respectively. The plurality of image sensors 104, 105 and 106 have different imaging and light-receiving characteristics. The image sensor 104 captures a red (R) wavelength region, the image sensor 105 captures a green (G) wavelength region, and the image sensor 106 captures a blue (B) wavelength region. A plurality of images captured by the plurality of image sensors 104, 105 and 106 are image-processed by an R signal processing circuit 107, a G signal processing circuit 108 and a B signal processing circuit 109, respectively, and combined and outputted as a color image by an image combining process circuit 110.
In this multiple-lens image pickup device, the plurality of imaging optical systems 101, 102 and 103 have different optical axes, and they are arranged to be symmetrically inclined at an angle θ (radiation angle) with respect to the normal line of the object placed at a predetermined position. For example, with the radiation angle θ being set and fixed to be optimal for the object position b of FIG. 15, if an object placed at the position a or c is captured, because the radiation angle θ for the object position a or c is different from the optimal radiation angle, a shift occurs among the images captured by the image sensors 104, 105 and 106.
This will be described with reference to FIGS. 16A, 16B and 16C. FIGS. 16A, 16B and 16C are diagrams showing combined images obtained by the multiple-lens image pickup device shown in FIG. 15. FIG. 16A is a combined image obtained when the object placed at the position a is captured. FIG. 16B is a combined image obtained when the object placed at the position b is captured. FIG. 16C is a combined image obtained when the object placed at the position c is captured. In this example, the object is assumed to be a white circular object captured on a black background.
When the object is placed at the position a, because the radiation angles of the plurality of imaging optical systems101, 102 and 103 are not appropriate, the red image (R) and the blue image (B) shift to the right and left from the green image as shown in FIG. 16A. The shifted portions are outputted as color drift in the combined image. More specifically, the portion from which the blue image has shifted becomes yellow (Ye) due to the combination of the green image and the red image. The portion from which the red image has shifted becomes cyan (Cy) due to the combination of the green image and the blue image. The portion from which the blue image and the red image have shifted becomes green (G). In this example, because the optical axes of the plurality of imaging optical systems 101, 102 and 103 are arranged one-dimensionally, in the combined image, a one-dimensional shift occurs along the arrangement direction, but when they are arranged two-dimensionally, a two-dimensional shift occurs. When a two-dimensional shift occurs in a combined image, the portion from which the green image has shifted becomes magenta (Mg) due to the combination of the red image and the blue image.
When the object is placed at the position b, because the radiation angles of the plurality of imaging optical systems101, 102 and 103 are appropriate, a high definition image without color drift as shown in FIG. 16B is outputted.
When the object is placed at the position c, a combined image as shown in FIG. 16C is obtained in which the red image and the blue image are shifted in the opposite direction to that of the combined image (see FIG. 16A) obtained when the object is placed at the position a.
Moreover, if there is even a slight difference (e.g., a variation in magnification, inclination, distortion, etc.) among a plurality of images obtained by the image sensors 104, 105 and 106, it will be difficult to correct the parallax accurately, and the image quality, particularly, the resolution of the combined image, will be very poor.
To cope with this, Japanese Patent No. 3397758 discloses, in order to prevent the variation in magnification, to set the focal length of a first imaging optical system corresponding to a first wavelength and that of a second imaging optical system corresponding to a second wavelength to be equal. However, from the actual production point of view, it would be extremely difficult to set the focal lengths to be exactly equal, and particularly when the optical systems correspond to different wavelength bands, it is essentially impossible.